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Abstract. Instructors have to make important decisions regarding the type of 
assessment that will finally be implemented when producing online learning 
materials, since this has a noticeable effect in terms of the methodology, the 
approach, and the attitude of all the actors involved (Goertler, 2011). This study 
investigates students’ perception of tutor assessment and self-assessment in an 
online upper-intermediate level English course (described in Martinez-Saez, 2015) 
and testing tool (described in Sevilla Pavon, 2013). After the implementation of 
these learning resources through the online authoring tool and content manager 
InGenio*, 95 students participated in the study conducted for the validation stages. 
The statistical analysis of the data gathered by means of questionnaires showed that 
the students highly appreciated the freedom to learn at their own pace offered by 
these two modalities which can contribute to autonomous progress and learning. 
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1. Introduction 


Students’ attitudes and motivation are two components which play a central role in 
language learning (Grzib-Schlosky, 2002; Martinez-Saez, 2015). These features, 
along with learners’ specific goals, interests, needs, or expectations, were deemed 
key to strategic objectives during the production of the InGenio learning resources, 
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and have been the basis for the research described in this paper. The resulting 
courseware comprises an online course and a tester aimed at helping students 
track their own progress before facing any official upper-intermediate English 
exam — B2-level according to the Common European Framework of Reference for 
Languages (CEFRL). 


Upon designing and developing the courseware, research was conducted on the 
importance of assessment within the learning process and the theoretical distinction 
between several pedagogical strategies in an attempt to improve the learning and 
evaluation method offered to the final users. The authors explored how online 
authoring tools and content managers, and in particular, InGenio, contribute to 
effectiveness and efficiency when two modalities such as self-assessment and tutor 
assessment are integrated into the resulting resources in order to allow students 
to monitor their process. One of the main advantages reported was the flexibility 
and adaptability provided to the students by this type of system (Sevilla-Pavon, 
Martinez-Saez, & Macario de Siqueira, 2011). 


In the design and development stages, special attention was paid to the variants 
known as tutor assessment, i.e. an autonomous online learning process monitored 
by a lecturer, and self-assessment, i.e. an autonomous online learning process in 
which students receive previously designed automatic feedback, hints, as well as 
model answers (both in the written and oral sections). The first modality “leads to 
the development of the students’ autonomy and sense of responsibility over the 
learning process, while enabling students to get as much help and support as they 
need from a human tutor in order to both complete the different tasks and attain the 
target level of language” (Sevilla-Pavon et al., 2011, p. 47). The second modality 
“enables students to conduct their own learning process and to assess their own 
learning achievements in an independent and autonomous way” (Sevilla-Pavon et 
al., 2011, p. 47). These were the two learning and evaluation methods which were 
finally integrated into the courseware in order to respond to the highly demanding 
requirements and standards of an effective self-access online course (Gimeno- 
Sanz, 2009; Levy & Stockwell, 2006). 


2. Method 


This paper addresses a specific part of the InGenio Course and Tester validation 
study conducted throughout two consecutive academic years (2011-12 and 
2012-13). A total number of 95 undergraduate students took part in it. They had 
previously enrolled on the course ‘Computer-Assisted English’ (4.5 ECTS), an 
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online subject offered at Universitat Politécnica de Valéncia. All students were 
granted access to the InGenio FCE Online Course & Tester (Module 1)° and were 
asked to complete all their contents in one semester. Therefore, all the students 
engaged in both tutoring assessment and self-assessment, i.e. the two learning and 
evaluation variants finally incorporated into the resulting courseware. 


Two different questionnaires (available from Martinez-Saez, 2015) were 
designed so as to monitor students’ progress and perceptions of a wide variety of 
components before (pre-course survey) and after (post-course survey) completing 
the first module of the Course and Tester. Among the components measured by 
these surveys administered through Google Forms, the following stand out: (1) 
length, usefulness, and effectiveness of the theoretical and practical contents; (2) 
the templates provided by the online authoring tool; (3) feedback; (4) interaction 
between the user and the platform; and (5) the range of assessment options. 


Thus, in this part of the study, the participants rated and evaluated the above-mentioned 
learning and assessment modalities. They also compared them with other learning 
scenarios in order to determine which alternative would best suit their needs and 
interests. In so doing, the fourth part of the pre-course survey was entitled ‘Learning 
styles’ and included two multiple-choice and two open-ended questions. As for the 
post-course survey, it was based on a 7-point Likert scale and included several enquiry 
items relating to learning and assessment modalities. These items were queried in 
two different sections: ‘General features’ and ‘Student self-assessment and personal 
comments’, comprised of four and seven questions, respectively. 


Due to the significant amount of data retrieved through the two questionnaires, only 
the most outstanding findings relating to the aims of this paper will be included. 


3. Results and analysis 


3.1. Pre-course survey 


In the initial questionnaire, the 95 students had to decide on the best way to learn a 
language; their preference went for a Face-to-Face (F2F) course in the classroom 
with a teacher (see Figure |). These preferences can be compared with the results 
displayed in Figure 3. 


5. The first module of the Course and Tester was implemented in 2011. 
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Figure 1. The best way to learn a language is... (pre-course survey) 


@ Option A: A face-to-face course in 
the classroom with a teacher 


™ Option B: A face-to-face course 
using technology in the classroom 


®™ Option C: An online course being 
assessed by a teacher 


H Option D: A self-access online 
course working by yourself 


Figure 2 illustrates the participants’ perceptions when asked about the most 
appropriate assessment modality in that particular online learning context. In the 
open-ended questions, a high number of students reported that flexibility is seen 
as an essential component when learning a language and that they highly valued 
feedback provision. 


Figure 2. Which learning modality would you prefer? 


™ Option A: Self-access learning 
(learning online without a tutor) 


™ Option B: Autonomous learning 
(guided by a tutor) 


® Option C: Blended learning 
(combining F2F sessions with 
autonomouslearning) 


3.2. Post-course survey 


A total number of 49 students completed all the Course and Tester contents and 
tasks and participated in this survey. In Section A, they rated four factors relating 
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to the learning and assessment modalities. As shown in Table |, the participants 
reported that “the Course and Tester encouraged autonomous / independent 
learning” (Item 17). A high number also emphasised how “they enjoyed having 
freedom to organise their time their own way” (Item 19) and “being able to self- 
assess their progress by accessing the assessment reports” (Item 21). Furthermore, 
the feedback they received was seen as very useful and relevant by most of the 
students. 


Table 1. Section A (Likert scale) 


1 2 3 4 5 6 ai 
Item 17 6% 12% 33% 39% 10% 
Item 19 2.5% 2.5% 16% 12% 20% 47% 
Item 21 2% 14% 31% 29% 24% 


Section C, the third part of the post-course questionnaire, provided evidence of 
students’ very positive attitude towards the use of technology and, in particular, 
to online environments after completing the Course and Tester. They also found it 
very easy to work autonomously. 


Figure 3 illustrates that the students’ general perception of the best modality to 
join a language course had experienced some changes when compared with the 
same question in the first questionnaire. The percentages of Options C and D were 
still very low in the final survey. However, there was an increase of 11% in the 
number of students who opted for Option B, which had a clear negative impact on 
Option A. 


Figure 3. The best way to learn a language is... (post-course survey) 


™ Option A: A face-to-face course in 
the classroom with a teacher 


@ Option B: A face-to-face course 
using technology in the classroom 


® Option C: An online course being 
assessed by a teacher 


@ Option D: A self-access online 
course working by yourself 
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4. Conclusions 


As seen in the previous section, students’ perceptions of the courseware and 
most of the components were very positive. An outstanding number welcomed 
features such as being able to self-assess their own progress and felt comfortable 
working in an online environment. However, when asked about their general 
perception of the best modality to take a language course, a significant number 
of learners still reported a preference for a hypothetical blended-learning 
alternative combining the F2F classroom component being assisted by a 
lecturer or tutor with an appropriate use of the online resources provided. Even 
though technology, flexibility, and adaptability were seen as fundamental and 
motivating features, the support given by the human component was highly 
appreciated. 
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